\chapter{TOUGH2 listing files}
\label{listingfiles}

\section{Introduction}
The \texttt{t2listing} library in PyTOUGH contains classes and routines for reading TOUGH2 listing files.  It can be imported using the command:

\begin{lstlisting}
   from t2listing import *
\end{lstlisting}

\index{TOUGH2}
\index{TOUGH2!AUTOUGH2}
\index{TOUGH2!TOUGH2-MP}
\index{TOUGH2!TOUGH+}
\index{TOUGH2!TOUGHREACT}

Listing files produced by AUTOUGH2, TOUGH2, TOUGH2\_MP, TOUGH+ and TOUGH3 have different formats but are all supported.  The main listing files produced by TOUGHREACT are also supported. (There is also a separate \hyperref[toughreact_tecplot]{\texttt{toughreact\_tecplot}} class for handling the additional Tecplot output files produced by TOUGHREACT.)

\section{\texttt{t2listing} objects}
\index{PyTOUGH!classes!\texttt{t2listing}}
\index{TOUGH2 listing files!objects}
\index{TOUGH2 listing files!creating}

The \texttt{t2listing} library defines a \texttt{t2listing} class, used for representing TOUGH2 listing files.

\textbf{Example:}

\begin{lstlisting}
lst = t2listing()
\end{lstlisting}

creates an empty \texttt{t2listing} object called \texttt{lst}.

\begin{lstlisting}
lst = t2listing(filename)
\end{lstlisting}

creates a \texttt{t2listing} object called \texttt{lst} and reads its contents from file \texttt{filename}.

\subsection{Properties}
\label{t2listing_properties}
\index{TOUGH2 listing files!properties}

The main properties of a \texttt{t2listing} object are listed in Table \ref{tb:t2listing_properties}.

\subsubsection{Element, connection and generation tables}
\index{TOUGH2 listing files!tables}

There are three main `table' properties, corresponding to the \textbf{element}, \textbf{connection} and \textbf{generation} tables in the listing file.  These are all of type \hyperref[listingtableobjects]{\texttt{listingtable}} (see section \ref{listingtableobjects}) and provide access to the simulation results.  Not all of these tables will necessarily be present - this depends on the settings in the data file which produced the results.  For TOUGH2 results, a fourth \textbf{primary} table may also be present, containing primary variables and their changes, if the KDATA parameter is set to 3.  \index{TOUGH2!TOUGH+} TOUGH+ results can also contain additional element tables containing other calculated quantities; these are named \textbf{element1}, \textbf{element2} etc.  A list of names of all available tables is given by the \texttt{table\_names} property.

For example, for a \texttt{t2listing} object \texttt{lst}, \texttt{lst.element['AR210']['Temperature']} gives the temperature at block `AR210', at the current time.  Blocks can also be identified by index rather than name, so that \texttt{lst.element[120]['Pressure']} gives the pressure at the block with (zero-based) index 120.

These tables can also be accessed to give all results for a given block, or for a given column in the table.  For example, \texttt{lst.element['AR210']} returns a dictionary containing all results at block `AR210', referred to by the name of each table column.  \texttt{lst.element['Temperature']} returns an \texttt{np.array} containing the temperatures at all blocks in the model.  (Hence, \texttt{lst.element['Pressure'][120]} gives the same result as \texttt{lst.element[120]['Pressure']}.)

The connection and generation tables work very similarly to the element table, except that connections are referred to by tuples of block names (rather than single block names), and generators are referred to by tuples of block names and generator names.  So for example, the mass flow rate between blocks `AB300' and `AC300' might be given by \texttt{lst.connection['AB300', 'AC300']['Mass flow']}.

The names of the columns for each table are read directly from the listing file, and will depend on the TOUGH2 equation of state (EOS) being used.

\subsubsection{Skipping tables}
\index{TOUGH2 listing files!tables!skipping}

The default behaviour is for a \texttt{t2listing} object to read all tables present in the listing file.  However, it is possible to skip the reading of specified tables if required.  This can be useful for speeding up reading of large listing files where not all tables are required.  For example, sometimes the connection data are not required, but for large models the connection table is often much bigger than the others, so skipping it can make reading significantly faster.  Data in skipped tables are not available either via their corresponding properties or via the \hyperref[sec:t2listing:history]{\texttt{history()}} method.

To skip tables, specify their table names (\texttt{element}, \texttt{connection} etc.) in the optional \texttt{skip\_tables} parameter when creating the \texttt{t2listing} object.  (By default, this parameter is an empty list.)  For example, to read a listing file with name `output.listing' into the object \texttt{lst} and skip reading the connection and generation tables:

\begin{lstlisting}
lst = t2listing('output.listing', skip_tables = ['connection', 'generation'])
\end{lstlisting}

\subsubsection{File encoding}
\index{TOUGH2 listing files!file encoding}

It is possible to specify the file encoding for the listing file using the optional \texttt{encoding} parameter when creating the \texttt{t2listing} object. The default for this parameter is ``latin-1'' encoding which should be fine for reading in most listing files. If you encounter exotic characters in your listing files which are not read correctly using the default encoding you may want to try other encodings.

\subsubsection{Full and short output}
\index{TOUGH2 listing files!short output}

AUTOUGH2 allows the use of `short' output, in which a specified selection of block, connection or generator properties are printed at time steps between normal full output.  A \texttt{t2listing} object will read short output results, if they are present, when producing time histories using the \hyperref[sec:t2listing:history]{\texttt{history()}} method.  However it is not possible to navigate to short output results or access them via the \texttt{t2listing} table properties above.

TOUGH2, TOUGH2\_MP, TOUGHREACT, TOUGH+ and TOUGH3 do not support short output.

\subsubsection{Navigating in time using \texttt{time}, \texttt{index} and \texttt{step}}
\index{TOUGH2 listing files!navigating in time}

The \texttt{time} property returns the time (in seconds) corresponding to the current set of results.  It is also possible to set the \texttt{time} property to navigate to a specific set of full results.  For example, \texttt{lst.time=1.e9} navigates to the set of full results with time closest to $10^9$s.

The \texttt{index} property gives the index of the current set of results, and can take any value between 0 and \texttt{num\_fulltimes}-1.  The value of \texttt{index} can also be set to change to a different set of results in the listing file (e.g. \texttt{lst.index=12}).  It can be incremented and decremented like any other Python integer variable, e.g. \texttt{lst.index+=1} or \texttt{lst.index-=2} to go to the next set of results, or the second to last set respectively.

The \texttt{step} property gives the time step number for the current set of results.  This is the number of time steps carried out in the simulation up to the current set of results (recall that results are not necessarily written to the listing file at every time step).  Again, its value can be set to navigate through the results, e.g. \texttt{lst.step=100} navigates to the set of full results with time step number nearest to 100.

The \texttt{times} property returns an \texttt{np.array} of all times at which results (including short output) are given in the listing file.  It has length equal to \texttt{num\_times}.  The \texttt{fulltimes} property returns an \texttt{np.array} of times at which full results are given (not including short output), and has length equal to \texttt{num\_fulltimes}.

A \texttt{t2listing} object also has methods (as well as properties) for navigating through time (see section \ref{t2listingmethods}).

\subsubsection{Listing diagnostics}
\index{TOUGH2 listing files!diagnostics}

\texttt{t2listing} objects have two properties that provide diagnostics on the results of the TOUGH2 run.

The \texttt{convergence} property is a dictionary of the maximum absolute differences in the element table between the second to last and last sets of results in the listing file.  This can be used to check convergence of steady-state simulations.  For example:

\begin{lstlisting}
lst.convergence['Temperature']
\end{lstlisting}

gives the largest absolute temperature change between the second to last and last sets of results.

The \texttt{reductions} property is a list of tuples of time step indices at which the time step size was reduced during the simulation, and the block name at which the maximum residual occurred prior to each reduction.  This gives an indication of problematic times and blocks which caused time step reductions.

\index{TOUGH2 listing files!properties}
\begin{table}
  \begin{center}
    \begin{tabular}{|l|l|p{70mm}|}
      \hline
      \textbf{Property} & \textbf{Type} & \textbf{Description}\\
      \hline
      \texttt{connection} & \hyperref[listingtableobjects]{\texttt{listingtable}} & connection table for current set of results\\
      \texttt{convergence} & dictionary & maximum differences in element table between second to last and last sets of results\\
      \texttt{element} & \hyperref[listingtableobjects]{\texttt{listingtable}} & element table for current set of results\\
      \texttt{element1} etc. & \hyperref[listingtableobjects]{\texttt{listingtable}} & additional element table for current set of results (TOUGH+ only)\\
      \texttt{filename} & string & name of listing file on disk\\
      \texttt{fullsteps} & \texttt{np.array} & array of time step numbers (integer) for full results\\
      \texttt{fulltimes} & \texttt{np.array} & array of times (float) for full results\\
      \texttt{generation} & \hyperref[listingtableobjects]{\texttt{listingtable}} & generation table for current set of results\\
      \texttt{index} & integer & index of current set of results\\
      \texttt{num\_fulltimes} & integer & number of sets of full results\\
      \texttt{num\_times} & integer & number of sets of all results (full and short)\\
      \texttt{primary} & \hyperref[listingtableobjects]{\texttt{listingtable}} & primary variable table for current set of results (TOUGH2 only)\\
      \texttt{reductions} & list & time step indices at which time step was reduced during the simulation\\
      \texttt{short\_types} & list of string & types of short output present\\
      \texttt{simulator} & string & detected simulator (`AUTOUGH2', `TOUGH2' etc.)\\
      \texttt{step} & integer & time step number of current set of results\\
      \texttt{steps} & \texttt{np.array} & array of time step numbers (integer) for all results (full and short)\\
      \texttt{table\_names} & list & names of available tables\\
      \texttt{time} & float & time of current set of results\\
      \texttt{times} & \texttt{np.array} & array of times (float) for all results (full and short)\\
      \texttt{title} & string & simulation title\\
      \hline
    \end{tabular}
    \caption{Properties of a \texttt{t2listing} object}
    \label{tb:t2listing_properties}
  \end{center}
\end{table}

\subsection{Methods}
\label{t2listingmethods}

The main methods of a \texttt{t2listing} object are listed in Table \ref{tb:t2listing_methods}.  Details of these methods are given below.

\index{TOUGH2 listing files!methods}
\begin{table}
  \begin{center}
    \begin{tabular}{|l|l|p{80mm}|}
      \hline
      \textbf{Method} & \textbf{Type} & \textbf{Description}\\
      \hline
      \hyperref[sec:t2listing:add_side_recharge]{\texttt{add\_side\_recharge}} & -- & adds side recharge generators to a \texttt{t2data} object\\
      \hyperref[sec:t2listing:close]{\texttt{close}} & -- & closes listing file\\
      \hyperref[sec:t2listing:first]{\texttt{first}} & -- & navigates to the first set of full results\\
      \hyperref[sec:t2listing:get_difference]{\texttt{get\_difference}} & dictionary & maximum differences in element table between two sets of results\\
      \hyperref[sec:t2listing:history]{\texttt{history}} & list or tuple & time history for a selection of locations and table columns\\
      \hyperref[sec:t2listing:last]{\texttt{last}} & -- & navigates to the last set of full results\\
      \hyperref[sec:t2listing:next]{\texttt{next}} & Boolean & navigates to the next set of full results\\
      \hyperref[sec:t2listing:prev]{\texttt{prev}} & Boolean & navigates to the previous set of full results\\
      \hyperref[sec:t2listing:write_vtk]{\texttt{write\_vtk}} & -- & writes results to VTK file\\
      \hline
    \end{tabular}
    \caption{Methods of a \texttt{t2listing} object}
    \label{tb:t2listing_methods}
  \end{center}
\end{table}

\begin{snugshade}
\subsubsection{\texttt{add\_side\_recharge(\emph{geo}, \emph{dat})}}
\end{snugshade}
\label{sec:t2listing:add_side_recharge}
\index{TOUGH2 listing files!using to add side recharge}

Adds side recharge generators to a \texttt{t2data} object \texttt{dat} for a production run, calculated according to the final results in the listing.  These generators represent side inflows due to pressure changes in the blocks on the model's horizontal boundaries.  Recharge generators are given the names of their blocks- any existing generators with the same names will be overwritten.

\textbf{Parameters:}
\begin{itemize}
\item \textbf{geo}: \hyperref[mulgrids]{\texttt{mulgrid}}\\
  Geometry object associated with the listing.
\item \textbf{dat}: \texttt{t2data}\\
  TOUGH2 data object for the side recharge generators to be added to.
\end{itemize}

\begin{snugshade}
\subsubsection{\texttt{close()}}
\end{snugshade}
\label{sec:t2listing:close}
\index{TOUGH2 listing files!closing}

Closes the listing file after use.

\begin{snugshade}
\subsubsection{\texttt{first()}}
\end{snugshade}
\label{sec:t2listing:first}
\index{TOUGH2 listing files!navigating in time}

Navigates to the first set of full results in the listing file.

\begin{snugshade}
\subsubsection{\texttt{get\_difference(\emph{indexa}=None, \emph{indexb}=None)}}
\end{snugshade}
\label{sec:t2listing:get_difference}
\index{TOUGH2 listing files!tables!maximum differences}

Returns dictionary of maximum differences, and locations of difference, of all element table properties between two sets of results.

\textbf{Parameters:}
\begin{itemize}
\item \textbf{indexa}, \textbf{indexb}: integer or \texttt{None}\\
  Indices of results between which the maximum differences are to be calculated.  If both indexa and indexb are provided, the result is the difference between these two result indices.  If only one index is given, the result is the difference between the given index and the one before that.  If neither are given, the result is the difference between the last and penultimate sets of results.
\end{itemize}

\begin{snugshade}
\subsubsection{\texttt{history(\emph{selection},  \emph{short}=True, \emph{start\_datetime}=None})}
\end{snugshade}
\label{sec:t2listing:history}
\index{TOUGH2 listing files!time histories}

Returns a list of time histories (as \texttt{np.arrays}) for specified locations and table columns in the element, connection or generation tables.  For each selection, a tuple of two \texttt{np.arrays} is returned, one each for times and values.  Short output (AUTOUGH2 only) can be omitted from the history results by setting the \texttt{short} parameter to \texttt{False}. If the \texttt{start\_datetime} parameter is given, times in the output are given as datetimes rather than seconds from the start.

\textbf{Parameters:}
\begin{itemize}

\item \textbf{selection}: list of tuples\\
  Selection of listing tables, locations (or indices) and table columns to produce histories for.  Each tuple contains three elements: the listing \textbf{table type} (`e', `c', `p' or `g' for element, connection, primary or generation table respectively), the \textbf{block/ connection/ generator name} (or index) and the \textbf{table column name}.  (If only a single tuple is given instead of a list of tuples, just the single tuple of times and values for that selection is returned.)  For history of additional element tables in TOUGH+ results, use `e1', `e2' etc. instead of `e'.  Note that, as for listing tables, connection and generator names (or `keys') are specified as two-element tuples (see Table \ref{tb:listing_table_keys}). If the second element of a selection tuple is an integer, it will be interpreted as the (zero-based) index of the block, connection or generator in the corresponding table.

\item \textbf{short}: Boolean\\
  Whether short output (AUTOUGH2 only) is to be included in the history results - default is \texttt{True}.

\item \textbf{start\_datetime}: datetime or \texttt{None}\\
  Datetime of the start of the simulation. If \texttt{None} (the default), output times are given as seconds from the start of the simulation. If a Python datetime is given, then output times are given as datetimes.

\end{itemize}

\textbf{Examples:}

\begin{lstlisting}
[(tt,temp), (tq,q), (tg,g)] = lst.history([('e', 'AR210', 'Temperature'),
('c', ('AB300','AC300'), 'Mass flow'), ('g', ('BR110','SO  1'), 'Generation rate')])
\end{lstlisting}

returns a list of three tuples of \texttt{np.arrays}, \texttt{(tt,temp)}, \texttt{(tq,q)} and \texttt{(tg,g)}, giving the times and values of temperature at block `AR210', mass flow at the connection between blocks `AB300' and `AC300', and generation rate in the generator `SO  1' in block `BR110' respectively.

\begin{lstlisting}
  from datetime import datetime
  t0 = datetime(1955, 1, 1)
  t,T = lst.history(('e', 'AR210', 'Temperature'), start_datetime = t0)
\end{lstlisting}

returns \texttt{T} as an \texttt{np.array} of temperature values, and \texttt{t} as an \texttt{np.array} of Python datetimes, starting at 1 January 1955.

\begin{snugshade}
\subsubsection{\texttt{last()}}
\end{snugshade}
\label{sec:t2listing:last}
\index{TOUGH2 listing files!navigating in time}

Navigates to the last set of full results in the listing file.

\begin{snugshade}
\subsubsection{\texttt{next()}}
\end{snugshade}
\label{sec:t2listing:next}
\index{TOUGH2 listing files!navigating in time}

Navigates to the next set of full results in the listing file.  Returns \texttt{False} if already at the last set of results (and \texttt{True} otherwise).

\begin{snugshade}
\subsubsection{\texttt{prev()}}
\end{snugshade}
\label{sec:t2listing:prev}
\index{TOUGH2 listing files!navigating in time}

Navigates to the previous set of full results in the listing file.  Returns \texttt{False} if already at the first set of results (and \texttt{True} otherwise).

\begin{snugshade}
\subsubsection{\texttt{write\_vtk(\emph{geo}, \emph{filename}, \emph{grid}=None, \emph{indices}=None, \emph{flows}=False,
\emph{wells}=False,\\
\emph{start\_time}=0, \emph{time\_unit}='s', \emph{flux\_matrix}=None, \emph{blockmap} = \{\},\\
 \emph{surface\_snap}=0.1)}}
\end{snugshade}
\label{sec:t2listing:write_vtk}
\index{TOUGH2 listing files!writing!VTK files}
\index{Visualization Tool Kit (VTK)}

Writes a \texttt{t2listing} object to a set of VTK files on disk, for visualisation with VTK, Paraview, Mayavi etc.  The results in the listing object are written as an `unstructured grid' VTK object with data arrays defined on cells.  The data arrays written correspond to the variables given in the columns of the element table of the \texttt{t2listing} object.  (For TOUGH+ results, variables from the additional element tables are also included.) In addition, data arrays from an associated \texttt{mulgrid} and (optionally) \texttt{t2grid} objects can be included.

If \texttt{flows} is \texttt{True} (and a \texttt{grid} is specified and the listing contains connection data), approximate block-average flux vectors at the centre of each block are also written, for all variables in the connection table with names ending in `flow'.

One *.vtu file is produced for each time step in the \texttt{t2listing} object at which full results are present, and a *.pvd file is also written.  This is usually the file that should actually be opened in Paraview or other software as it contains time information associated with each *.vtu file.

Optionally, only a subset of the time indices present in the \texttt{t2listing} can be written, according to the \texttt{indices} parameter.  A start time and time unit for the output can optionally be specified.

\textbf{Parameters:}
\begin{itemize}
\item \textbf{geo}: \hyperref[mulgrids]{\texttt{mulgrid}}\\
  The \texttt{mulgrid} geometry object associated with the results.  For flexibility, this geometry need not be fully compatible with the results -- for example, it may contain only a subset of the blocks for which results are present, or the blocks may be in a different order.  However, if it is not fully compatible, the writing process will be slower, and flux vectors will not be written (even if \texttt{flows} is set to \texttt{True}).
\item \textbf{filename}: string\\
  Name of the *.pvd file to be written.  Names of the individual *.vtu files for each time step are similar but with a time index appended and the file extension changed.
\item \textbf{grid}: \hyperref[t2grids]{\texttt{t2grid}}\\
  Name of optional \texttt{t2grid} object associated with the results.
\item \textbf{indices}: list or tuple\\
  Optional specification of time indices to include in the output.  If set to \texttt{None} (the default), all time indices will be included.
\item \textbf{flows}: Boolean\\
  Set to \texttt{True} if approximate block-centred flux vectors are to be calculated and written, for visualising flows.  Default is \texttt{False}.  \textbf{Note}: flow vectors can only be calculated if a \textbf{grid} is specified.
\item \textbf{wells}: Boolean\\
  Set to \texttt{True} if a separate VTK file for the wells in the \hyperref[mulgrids]{\texttt{mulgrid}} object is to be written.  Default is \texttt{False}.
\item \textbf{start\_time}: float\\
  Optional start time of the simulation, i.e. time associated with the first set of results.  Default is zero.
\item \textbf{time\_unit}: string\\
  Optional time unit for the output.  TOUGH2 results are given at times in seconds, but this option allows them to be converted to other units.  Options are: `s', `h', `d' and `y', for seconds, hours, days and years respectively.  Default is `s'.
\item \textbf{flux\_matrix}: \texttt{scipy.sparse.lil\_matrix}\\
  Sparse matrix that multiplies a vector of connection values to produce a partition vector of 3-D block average flows at the (underground) block centres.  One of these can be produced using the \texttt{t2grid.flux\_matrix()} method, and a corresponding \texttt{mulgrid} object.  A flux matrix will be calculated internally if not supplied.
\item \textbf{blockmap}: dictionary\\
  Dictionary mapping the block names in the geometry to the block naming system used in the listing.
\item \textbf{surface\_snap}: float\\
  Tolerance for specifying how close column surface elevations need to be before being considered ``equal'' when constructing surface nodes.
\end{itemize}

\section{\texttt{listingtable} objects}
\label{listingtableobjects}
\index{PyTOUGH!classes!\texttt{listingtable}}
\index{TOUGH2 listing files!tables}

A \texttt{listingtable} object represents a table of results in a TOUGH2 listing file (whether it is an element, connection or generation table).  The column headings of the table are taken directly from the corresponding table in the listing file.  The rows of the table may be accessed either by (zero-based) index, or by the `key' for the table row, which depends on the table type (see Table \ref{tb:listing_table_keys}).

\index{TOUGH2 listing files!tables!keys}
\begin{table}[h]
  \begin{center}
    \begin{tabular}{|l|l|}
      \hline
      \textbf{Table type} & \textbf{Key}\\
      \hline
      \texttt{element} & block name\\
      \texttt{connection} & (block name 1, block name 2)\\
      \texttt{generation} & (block name, generator name)\\
      \hline
    \end{tabular}
    \caption{Keys for different listing table types}
    \label{tb:listing_table_keys}
  \end{center}
\end{table}

Hence, the value in the element table for a given block and column can be accessed by \texttt{lst.element[blockname][columnname]}, or by \texttt{lst.element[blockindex][columnname]} (for a \texttt{t2listing} object \texttt{lst}).  Note that for connection and generation tables, the keys are tuples of two strings.  For connection tables, the order of these two strings (the block names) is not important; if the listing file contains results for (block1, block2), then results for (block2, block1) can be accessed via the corresponding \texttt{listingtable} object (though the results will have the opposite sign to those in the file, as they will represent flows in the opposite direction).

The values for an entire row or column of the table can also be accessed, for example \texttt{lst.element[blockname]} gives the row in the table for a specified block, with the values arranged in a dictionary which can be accessed using the column names of the table (e.g. \texttt{lst.element['AR231']['Temperature']}).  This dictionary for each row also contains an additional \texttt{'key'} item which returns the key for that row.  Conversely, \texttt{lst.element[columnname]} gives the column in the table for a specified column name, with the values returned in an \texttt{np.array} (one value for each block in the grid, for an element table).

\subsection{\texttt{listingtable} properties}

The properties of a \texttt{listingtable} object are given in Table \ref{tb:listingtable_properties}.  The entire list of key values for a \texttt{listingtable} may be accessed via the \texttt{row\_name} property, which contains the key value for each row.  The column headings of the table can similarly be accessed via the \texttt{column\_name} list property.  The \texttt{num\_rows} and \texttt{num\_columns} properties of a \texttt{listingtable} object return the numbers of rows and columns respectively.  The \texttt{num\_keys} property just returns the number of keys used to identify each row - generally 1 for an element table and 2 for connection and generation tables.

\index{TOUGH2 listing files!tables!properties}
\begin{table}
  \begin{center}
    \begin{tabular}{|l|l|l|}
      \hline
      \textbf{Property} & \textbf{Type} & \textbf{Description}\\
      \hline
      \texttt{column\_name} & list & column headings\\
      \texttt{DataFrame} & pandas DataFrame & data in DataFrame format\\
      \texttt{num\_columns} & integer & number of columns \\
      \texttt{num\_keys} & integer & number of keys per row \\
      \texttt{num\_rows} & integer & number of rows \\
      \texttt{row\_name} & list & keys for each row \\
      \hline
    \end{tabular}
    \caption{Properties of a \texttt{listingtable} object}
    \label{tb:listingtable_properties}
  \end{center}
\end{table}

\subsection{Adding and subtracting}
\index{TOUGH2 listing files!tables!adding and subtracting}
It is possible to perform addition and subtraction operations on \texttt{listingtable} objects.  Subtraction can be useful, for example, when comparing results from different runs.  These operations can only be carried out when the row and column names of the two tables are identical.  The resulting table will have the same row and column names as the original tables, but will contain the element-wise sums or differences.

\subsection{Converting to DataFrames}

A \texttt{listingtable} object has a \texttt{DataFrame} property which returns the entire table in the form of a \texttt{pandas} (\url{http://pandas.pydata.org/}) DataFrame object. \texttt{pandas} is a Python library for data analysis, which you will need to have installed before you can use the \texttt{DataFrame} property. With \texttt{pandas} you can do advanced data analysis on your TOUGH2 results. See the \texttt{pandas} documentation for more details.

\subsection{\texttt{listingtable} methods}

\texttt{listingtable} objects have one method as described below.

\index{TOUGH2 listing files!tables!methods}
\index{TOUGH2 listing files!tables!finding rows}
\begin{snugshade}
\subsubsection{\texttt{rows\_matching(\emph{pattern}, \emph{index}=0, \emph{match\_any}=False)}}
\end{snugshade}

Returns a list of rows in the table with keys matching the specified regular expression string, \texttt{pattern}.

For tables with multiple keys, \texttt{pattern} can be a list or tuple of regular expressions.  If a single string pattern is given for a multiple-key table, the pattern is matched on the index$^{th}$ key (and any value of the other key - unless the \texttt{match\_any} option is used; see below).

If \texttt{match\_any} is set to \texttt{True}, rows are returned with keys matching any of the specified patterns (instead of all of them).  If this option is used in conjunction with a single string pattern, the specified pattern is applied to all keys.

\textbf{Parameters:}
\begin{itemize}
\item \textbf{pattern}: string, list or tuple\\
  Regular expression string specifying the pattern to match.  For multiple-key tables, this can be a list or tuple of regular expression strings.
\item \textbf{index}: integer\\
  Index of the key to which the pattern is to be applied, for multiple-key tables and when \texttt{pattern} is specified as a single string.
\item \textbf{match\_any}: Boolean\\
  If \texttt{False}, return only rows with keys matching \emph{all} of their corresponding patterns.  If \texttt{True}, return rows with keys matching \emph{any} of the specified patterns - and if a single string pattern is given, apply this to all keys.
\end{itemize}

\section{\texttt{t2historyfile} objects}
\index{PyTOUGH!classes!\texttt{t2historyfile}}
\index{TOUGH2 history files}
\index{TOUGH2 data files!FOFT}
\index{TOUGH2 data files!COFT}
\index{TOUGH2 data files!GOFT}

In addition to the main listing file, TOUGH2 can optionally produce extra files containing time history data from selected blocks, connections or generators, named \texttt{FOFT}, \texttt{COFT} and \texttt{GOFT} files respectively.  TOUGH+ can optionally name these files \texttt{Elem\_Time\_Series}, \texttt{Conx\_Time\_Series} and \texttt{SS\_Time\_Series} instead.  (AUTOUGH2 does not produce separate history files, but can instead produce `short output' at selected blocks, connections or generators within the listing file itself.)

The \texttt{t2listing} module contains a \texttt{t2historyfile} class for reading and manipulating these history files.  History files produced by TOUGH2, TOUGH2\_MP and TOUGH+ are supported, although they all have different formats.  The same class is used for FOFT, COFT and GOFT files.  A history file of any of these types can be opened using a command such as:

\index{TOUGH2 history files!creating}
\begin{lstlisting}
hist = t2historyfile(filename)
\end{lstlisting}

\index{TOUGH2!TOUGH2-MP}
where \texttt{\emph{filename}} is the name of the file.  It may contain wildcards (*) so that several files matching a pattern are read in to the same object.  This is useful for reading output from TOUGH2\_MP, which creates separate history files for each processor used in the calculation (e.g. \texttt{FOFT\_P.000}, \texttt{FOFT\_P.001}, etc.).  It is assumed that all files opened are however of the same type (FOFT, COFT or GOFT).

Once a history file has been read in, history results for a particular key (i.e. block, connection or generator) can be extracted.  For TOUGH2\_MP, the keys are the block names for FOFT files, tuples of block names for COFT files, and tuples of block names and source names for GOFT files.  For example:

\begin{lstlisting}
foft = t2historyfile('FOFT_P.*')
blockname = 'fmq20'
results = foft[blockname]
\end{lstlisting}

This will return a dictionary containing an \texttt{np.array} for each column in the file, indexed by the column name.  For example the temperature history at this block would be given by:

\begin{lstlisting}
temp = foft[blockname]['TEMPERATURE']
\end{lstlisting}

Results at a particular time can also be found:

\begin{lstlisting}
time = 3.156e7
result = foft[blockname, time]
\end{lstlisting}

Again, this will return a dictionary with one item for each column, but in this case each item is just a single floating point number instead of an array.

\index{TOUGH2}
For \textbf{TOUGH2} rather than TOUGH2\_MP, the keys are integer indices of blocks, connections or generators, rather than names or tuples of names.  Similarly, the column names are just integers.  This is because the key names and column names are not given in TOUGH2 history files.  Aside from these differences, they can be used in the same way as TOUGH2\_MP history files, for example:

\begin{lstlisting}
foft = t2historyfile('FOFT')
blkindex = 123
temp = foft[blkindex][1]
\end{lstlisting}

\index{TOUGH2!TOUGH+}
For \textbf{TOUGH+} connection and generator history files (\texttt{COFT} and \texttt{GOFT}, or \texttt{Conx\_Time\_Series} and \texttt{SS\_Time\_Series}), multiple connections and generators can be specified as usual in the TOUGH2 input data file, but individual results for them are not written to the history file.  Instead, the results for them are summed.  As a result, there are no `keys' as such for accessing individual results, and the \texttt{t2historyfile} works a little differently.  An array containing the data in each column can be accessed by specifying the column name, for example:

\begin{lstlisting}
ct = t2historyfile('Conx_Time_Series')
qh = ct['HeatFlow']
\end{lstlisting}

The properties of a \texttt{t2historyfile} object are given in Table \ref{tb:historyfile_properties}.

\index{TOUGH2 history files!properties}
\begin{table}
  \begin{center}
    \begin{tabular}{|l|l|l|}
      \hline
      \textbf{Property} & \textbf{Type} & \textbf{Description}\\
      \hline
      \texttt{column\_name} & list & column headings\\
      \texttt{key\_name} & list & names of keys\\
      \texttt{num\_times} & integer & number of times\\
      \texttt{num\_columns} & integer & number of data columns\\
      \texttt{num\_rows} & integer & total number of data (for all keys)\\
      \texttt{simulator} & string & detected simulator (`TOUGH2' or `TOUGH2\_MP')\\
      \texttt{times} & \texttt{np.array} & times at which results are given\\
      \texttt{type} & string & history type (`FOFT', `COFT' or `GOFT')\\
      \hline
    \end{tabular}
    \caption{Properties of a \texttt{t2historyfile} object}
    \label{tb:historyfile_properties}
  \end{center}
\end{table}

\section{\texttt{toughreact\_tecplot} objects}
\label{toughreact_tecplot}
\index{TOUGHREACT Tecplot files}
\index{PyTOUGH!classes!\texttt{toughreact\_tecplot}}
\index{TOUGH2!TOUGHREACT}

The \texttt{t2listing} library also defines a \texttt{toughreact\_tecplot} class, used for representing the additional Tecplot output files produced by TOUGHREACT.

\textbf{Example:}

\begin{lstlisting}
tp = toughreact_tecplot(filename, blocks)
\end{lstlisting}

creates a \texttt{toughreact\_tecplot} object called \texttt{tp} and reads its contents from file \texttt{filename}. The \texttt{blocks} object passed in as a second parameter specifies the block names (see \ref{toughreact_tecplot_blocknames}).

\subsection{Differences from \texttt{t2listing} objects}

A \texttt{toughreact\_tecplot} object is similar to a \hyperref[listingfiles]{\texttt{t2listing}} object in many respects. Apart from the need to specify the block names on creation (see \ref{toughreact_tecplot_blocknames}), the other main difference is that unlike a \texttt{t2listing} object, which usually contains several \texttt{listingtable} objects, a \texttt{toughreact\_tecplot} object contains only one: the \texttt{element} table. Because of this, when using the \texttt{history} method, tables need not be specified.

These Tecplot files do not contain any information about time step numbers, so \texttt{t2listing} properties like \texttt{step} and \texttt{steps} are not present in a \texttt{toughreact\_tecplot} object. There is also no \texttt{title} property, as this is not present in the Tecplot file.

There is also no `short' output in a Tecplot file, so a \texttt{toughreact\_tecplot} object does not have properties like \texttt{fulltimes}, as this would just be the same as the \texttt{times} property. There are also no diagnostic methods like \texttt{convergence} or \texttt{reductions}.

\subsection{Specifying block names}
\label{toughreact_tecplot_blocknames}
\index{TOUGHREACT Tecplot files!block names}

In the Tecplot file, results are not associated with block names, though they appear in the same order as in the TOUGH2 data file used to generate the results. To make results accessible by block name, a second parameter containing the block names must be specified when a \texttt{toughreact\_tecplot} object is created. This parameter is not optional. It can be either:

\begin{itemize}
\item a list of strings specifying the block names
\item a \hyperref[mulgrids]{\texttt{mulgrid}} geometry object
\item a \hyperref[t2grids]{\texttt{t2grid}} object
\end{itemize}

\subsection{Properties}
\index{TOUGHREACT Tecplot files!properties}

The main properties of a \texttt{toughreact\_tecplot} object are listed in Table \ref{tb:toughreact_tecplot_properties}. For more details, see the corresponding properties of the \hyperref[t2listing_properties]{\texttt{t2listing}} class.

\begin{table}
  \begin{center}
    \begin{tabular}{|l|l|p{70mm}|}
      \hline
      \textbf{Property} & \textbf{Type} & \textbf{Description}\\
      \hline
      \texttt{element} & \hyperref[listingtableobjects]{\texttt{listingtable}} & element table for current set of results\\
      \texttt{filename} & string & name of listing file on disk\\
      \texttt{index} & integer & index of current set of results\\
      \texttt{num\_times} & integer & number of sets of results\\
      \texttt{time} & float & time of current set of results\\
      \texttt{times} & \texttt{np.array} & array of times (float) for all results\\
      \hline
    \end{tabular}
    \caption{Properties of a \texttt{toughreact\_tecplot} object}
    \label{tb:toughreact_tecplot_properties}
  \end{center}
\end{table}

\subsection{Methods}
\index{TOUGHREACT Tecplot files!methods}

The methods of a \texttt{toughreact\_tecplot} object are listed in Table \ref{tb:toughreact_tecplot_methods}. Details of these methods are given below.

\begin{table}
  \begin{center}
    \begin{tabular}{|l|l|p{90mm}|}
      \hline
      \textbf{Method} & \textbf{Type} & \textbf{Description}\\
      \hline
      \hyperref[sec:toughreact_tecplot:close]{\texttt{close}} & -- & closes file\\
      \hyperref[sec:toughreact_tecplot:first]{\texttt{first}} & -- & navigates to the first set of full results\\
      \hyperref[sec:toughreact_tecplot:history]{\texttt{history}} & list or tuple & time history for a selection of locations and table columns\\
      \hyperref[sec:toughreact_tecplot:last]{\texttt{last}} & -- & navigates to the last set of full results\\
      \hyperref[sec:toughreact_tecplot:next]{\texttt{next}} & Boolean & navigates to the next set of full results\\
      \hyperref[sec:toughreact_tecplot:prev]{\texttt{prev}} & Boolean & navigates to the previous set of full results\\
      \hyperref[sec:toughreact_tecplot:write_vtk]{\texttt{write\_vtk}} & -- & writes results to VTK file\\
      \hline
    \end{tabular}
    \caption{Methods of a \texttt{toughreact\_tecplot} object}
    \label{tb:toughreact_tecplot_methods}
  \end{center}
\end{table}

\begin{snugshade}
\subsubsection{\texttt{close()}}
\end{snugshade}
\label{sec:toughreact_tecplot:close}
\index{TOUGHREACT Tecplot files!closing}

Closes the file after use.

\begin{snugshade}
\subsubsection{\texttt{first()}}
\end{snugshade}
\label{sec:toughreact_tecplot:first}
\index{TOUGHREACT Tecplot files!navigating in time}

Navigates to the first set of results in the Tecplot file.

\begin{snugshade}
\subsubsection{\texttt{history(\emph{selection})}}
\end{snugshade}
\label{sec:toughreact_tecplot:history}
\index{TOUGHREACT Tecplot files!time histories}

Returns a list of time histories (as \texttt{np.arrays}) for specified locations and table columns in the element table.  For each selection, a tuple of two \texttt{np.arrays} is returned, one each for times and values.

\textbf{Parameters:}
\begin{itemize}
\item \textbf{selection}: list of tuples\\
  Selection of locations (or indices) and table columns to produce histories for.  Each tuple contains two elements: \textbf{block name} and \textbf{table column name}.  (If only a single tuple is given instead of a list of tuples, just the single tuple of times and values for that selection is returned.)
\end{itemize}

\begin{snugshade}
\subsubsection{\texttt{last()}}
\end{snugshade}
\label{sec:toughreact_tecplot:last}
\index{TOUGHREACT Tecplot files!navigating in time}

Navigates to the last set of results in the Tecplot file.

\begin{snugshade}
\subsubsection{\texttt{next()}}
\end{snugshade}
\label{sec:toughreact_tecplot:next}
\index{TOUGHREACT Tecplot files!navigating in time}

Navigates to the next set of results in the Tecplot file.  Returns \texttt{False} if already at the last set of results (and \texttt{True} otherwise).

\begin{snugshade}
\subsubsection{\texttt{prev()}}
\end{snugshade}
\label{sec:toughreact_tecplot:prev}
\index{TOUGHREACT Tecplot files!navigating in time}

Navigates to the previous set of results in the Tecplot file.  Returns \texttt{False} if already at the first set of results (and \texttt{True} otherwise).

\begin{snugshade}
\subsubsection{\texttt{write\_vtk(\emph{geo}, \emph{filename}, \emph{grid}=None, \emph{indices}=None, \emph{start\_time}=0,\\
    \emph{time\_unit}='s', \emph{blockmap} = \{\}, \emph{surface\_snap}=0.1)}}
\end{snugshade}
\label{sec:toughreact_tecplot:write_vtk}
\index{TOUGHREACT Tecplot files!writing!VTK files}
\index{Visualization Tool Kit (VTK)}

Writes a \texttt{toughreact\_tecplot} object to a set of VTK files on disk, for visualisation with VTK, Paraview, Mayavi etc.  The results in the element table of the Tecplot file object are written as an `unstructured grid' VTK object with data arrays defined on cells.  The data arrays written correspond to the variables given in the columns of the element table of the \texttt{toughreact\_tecplot} object. In addition, data arrays from an associated \texttt{mulgrid} and (optionally) \texttt{t2grid} objects can be included.

One *.vtu file is produced for each time step in the \texttt{toughreact\_tecplot} object, and a *.pvd file is also written.  This is usually the file that should actually be opened in Paraview or other software as it contains time information associated with each *.vtu file.

Optionally, only a subset of the time indices present in the \texttt{toughreact\_tecplot} can be written, according to the \texttt{indices} parameter.  A start time and time unit for the output can optionally be specified.

\textbf{Parameters:}
\begin{itemize}
\item \textbf{geo}: \hyperref[mulgrids]{\texttt{mulgrid}}\\
  The \texttt{mulgrid} geometry object associated with the results.  For flexibility, this geometry need not be fully compatible with the results -- for example, it may contain only a subset of the blocks for which results are present, or the blocks may be in a different order.  However, if it is not fully compatible, the writing process will be slower.
\item \textbf{filename}: string\\
  Name of the *.pvd file to be written.  Names of the individual *.vtu files for each time step are similar but with a time index appended and the file extension changed.
\item \textbf{grid}: \hyperref[t2grids]{\texttt{t2grid}}\\
  Name of optional \texttt{t2grid} object associated with the results.
\item \textbf{indices}: list or tuple\\
  Optional specification of time indices to include in the output.  If set to \texttt{None} (the default), all time indices will be included.
\item \textbf{start\_time}: float\\
  Optional start time of the simulation, i.e. time associated with the first set of results.  Default is zero.
\item \textbf{time\_unit}: string\\
  Optional time unit for the output.  TOUGHREACT Tecplot results are given at times in years, but this option allows them to be converted to other units.  Options are: `s', `h', `d' and `y', for seconds, hours, days and years respectively.  Default is `s'.
\item \textbf{blockmap}: dictionary\\
  Dictionary mapping the block names in the geometry to the block naming system used in the Tecplot output.
\item \textbf{surface\_snap}: float\\
  Tolerance for specifying how close column surface elevations need to be before being considered ``equal'' when constructing surface nodes.
\end{itemize}

\section{Examples}
\index{examples!TOUGH2 listing files}

\subsection{Slice plot of drawdown}

This script shows a vertical slice plot along the model's \emph{x}-axis of the difference in pressure (i.e. drawdown) between the start and end of a simulation.

\begin{lstlisting}
from mulgrids import *
from t2listing import *
from copy import copy

geo = mulgrid('gmodel.dat')
results = t2listing('model.listing')

results.first()
p0 = copy(results.element['Pressure'])
results.last()
p1 = results.element['Pressure']

geo.slice_plot('x', (p1-p0)/1.e5, 'Pressure\ difference', 'bar')
\end{lstlisting}

(Note: the \texttt{copy} command is needed, otherwise the arrays \texttt{p0} and \texttt{p1} would both contain the final values of pressure after the \texttt{results.last()} command.)

\subsection{Pressure-temperature diagram}

This script plots model results from a specified block on a pressure-temperature diagram.

\begin{lstlisting}
from t2listing import *
import matplotlib.pyplot as plt

lst = t2listing('model.listing')
blk = ' n 60'
[(tp,p), (tt,t)] = lst.history([('e', blk, 'Pressure'), ('e', blk, 'Temperature')])

plt.plot(t, p/1.e5, 'o-')
plt.xlabel('T ($\degree$C)')
plt.ylabel('P (bar)')
plt.show()
\end{lstlisting}

\subsection{Comparing results of two models}
\label{comparison_example}

This script reads grids and results for two different models, a coarse model and a fine one, and produces a comparison plot of the time history of temperature for both models at a given point.

\begin{lstlisting}
from mulgrids import *
from t2listing import *
import matplotlib.pyplot as plt

geoc, geof = mulgrid('gcoarse.dat'), mulgrid('gfine.dat')
coarse, fine = t2listing('coarse.listing'), t2listing('fine.listing')

p = [47.e3, 0.0, -7000.0]
blkc = geoc.block_name_containing_point(p)
blkf = geof.block_name_containing_point(p)

tc, tempc = coarse.history(('e', blkc, 'Temperature'))
tf, temp = fine.history(('e', blkf, 'Temperature'))

plt.plot(tc, tempc, 'o-', label = 'coarse model')
plt.plot(tf, tempf, 's-', label = 'fine model')
plt.xlabel('time (s)')
plt.ylabel('Temperature ($\degree$C)')
plt.legend()

plt.show()
\end{lstlisting}

